-25- 



The invention in which an exclusive right is claimed is defined by the following: 

1. A method of detecting a three-dimensional object adjacent to a 
user-interactive side of a surface that optically diffuses light, as a fimction of an 
infrared light transmitted toward the three-dimensional object from an opposite side 
of the surface and reflected back through the surface from the three-dimensional 
object to be received by a light sensor disposed on the opposite side of the surface, 
comprising the steps of: 

(a) creating a first pixilated image representing the intensity of the 
infrared light reflected from the three-dimensional object and received by the light 
sensor; 

(b) creating a first binarized image from the first pixilated image 
by filtering out pixels of the first pixilated image that do not have an intensity 
exceeding a first threshold value, the first binarized image representing a first planar 
distance of the three-dimensional object from the interactive side of the surface; 

(c) creating a second binarized image from the first pixilated 
image by filtering out pixels of the first pixilated image that do not have an intensity 
exceeding a second threshold value, the second binarized image having substantially 
equal area and coordinate locations as the first binarized image, the second binarized 
image representing a second planar distance from the interactive side of the surface; 

(d) detecting a first connected component in the first binarized 
image, the first connected component representing a first set of pixels that have an 
intensity exceeding the first threshold value and are immediately adjacent to each 
other, without an intervening region of pixels that do not have an intensity exceeding 
the first threshold value; 

(e) determining a first bounding area and a first coordinate 
location of the first connected component in the first binarized image; 

(f) detecting a second connected component in the second 
binarized image, the second connected component representing a second set of pixels 
that have an intensity exceeding the second threshold value and are immediately 
adjacent to each other, without an intervening region of pixels that do not have an 
intensity exceeding the second threshold value; 
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(g) determining a second bounding area and a second coordinate 
location of the second connected component in the second binarized image that has 
substantially equal area and coordinate locations as the first binarized image; and 

(h) determining that both the first connected component and the 
second connected component correspond to the three-dimensional object adjacent to 
the interactive side of the surface, if one of the first connected component and the 
second connected component is disposed substantially within the boxmding area of 
the other of the first connected component and the second connected component. 

2. The method of Claim 1, further comprising the steps of: 

(a) associating a first label with the first connected component; 

(b) associating a second label with the second connected 
component; and 

(c) associating the first label with the second label to indicate that 
both the first connected component and the second connected component correspond 
to the three-dimensional object that is adjacent to the interactive side of the surface. 

3. The method of Claim 1, further comprising the steps of: 

(a) placing the three-dimensional object on the surface prior to 
creating the first pixilated image, such that the first planar distance is substantially 
zero; and 

(b) after creating the first pixilated image, determining a touch 
threshold value for pixels that should be included in the first pixilated image, 
corresponding to an intensity of the infrared light reflected fi-om a portion of the 
three-dimensional object and received by the light sensor when said portion of the 
three-dimensional object is contacting the surface. 

4. The method of Claim 3, further comprising the step of setting the first 
threshold value to the touch threshold value prior to creating the first binarized image, 
thereby determining whether any portion of the three-dimensional object is contacting 
the surface. 
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5. The method of Claim 3, further comprising the steps of: 

(a) determining a hover threshold that is different from the touch 
threshold, the hover threshold corresponding to an intensity of the infrared light 
reflected by the three-dimensional object and received by the light sensor when the 
three-dimensional object is a determinable distance from the surface on the 
interactive side of the surface; 

(b) prior to creating the second binarized image, setting the second 
threshold value to the hover threshold; and 

(c) determining that the three-dimensional object is at least within 
a hover region away from the surface, the hover region comprising a volume between 
the surface and a plane that is parallel to the surface and is separated from the surface 
on the interactive side of the surface by the determinable distance. 

6. The method of Claim 5, wherein the step of determining the hover 
threshold is performed prior to creating the first binarized image and the second 
binarized image, the step of determining the hover threshold comprising the steps of: 

(a) moving the three-dimensional object toward the surface on the 
interactive side of the surface from a distance sufficiently far from the surface so that 
an intensity of the infrared light reflected from the three-dimensional object and 
received by the light sensor initially is not above the first threshold value; 

(b) creating a series of pixilated images over time as the 
three-dimensional object is moved toward the surface; 

(c) detecting a plurality of calibration connected components, 
including one calibration connected component corresponding to the 
three-dimensional object for each of the series of pixilated images, each calibration 
connected component comprising a set . of immediately adjacent pixels with a 
corresponding minimum pixel intensity for the infrared light received by the light 
sensor that is greater than zero for each of the sequence of pixilated images; 

(d) determining which one of the plurality of calibration connected 
components has a maximum bounding area; and 

(e) storing as the hover threshold the corresponding minimimi 
pixel intensity that was determined for said one of the plurality of calibration 
connected components that has the maximum bounding area. 
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7, The method of Claim 5, wherein the determinable distance is a 
function of an infrared light reflectivity property of the three-dimensional object. 

8. The method of Claim 5, further comprising the steps of: 

(a) creating a second pixilated image an interval of time after 
creating the first pixilated image; 

(b) creating a third binarized image from the second pixilated 
image by filtering out pixels of the second binarized image that do not have an 
intensity exceeding the hover threshold, so that the second binarized image and the 
third binarized image comprise a sequence of binarized images at the hover threshold; 

(c) detecting a third connected component in the third binarized 

image; 

(d) determining at least one of a third bounding area and a third 
location of the third connected component in the third binarized image; and 

(e) computing at least one of a current distance from the surface, a 
change in distance, and a velocity of the three-dimensional object as a function of at 
least two of the second bounding area, the third bounding area, the second location, 
and the third location. 

9, The method of Claim 8, further comprising the steps of: 

(a) predicting whether the three-dimensional object will contact 
the surface within a predefined period; and 

(b) producing an indication that the three-dimensional object has 
already contacted the surface if the prediction indicates that the three-dimensional 
object will contact the surface within the predefined period, thereby reducing latency 
in providing the indication that the three-dimensional object has contacted the 
surface. 

10. A memory medium on which are stored machine executable 
instructions for carrying out the steps of Claim 1. 
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11. A system for detecting a relative position of a three-dimensional 
object, the system comprising: 

(a) a surface that diffuses light and has: 

(i) an interactive side adjacent to v^hich the 
three-dimensional object can be manipulated; and 

(ii) an opposite side that is opposite the interactive side; 

(b) a light source spaced away the opposite side of the surface, the 
light source emitting an infrared light that is transmitted through the surface to the 
interactive side of the surface; 

(c) a light sensor disposed on the opposite side of the surface so as 
to sense infrared light reflected back from the three-dimensional object, through the 
surface; 

(d) a processor in communication with the light sensor; and 

(e) a memory in communication with the processor, the memory 
storing data and machine instructions that when executed by the processor cause a 
plurality of ftinctions to be carried out, including: 

(i) creating a first pixilated image representing the 
intensity of the mfrared light reflected from the three-dimensional object and received 
by the light sensor; 

(ii) creating a first binarized image from the first pixilated 
image by filtering out pixels of the first pixilated image that do not have an intensity 
exceeding a first threshold value, the first binarized image representing a first planar 
distance of the three-dimensional object from the interactive side of the surface; 

(iii) creating a second bmarized image from the first 
pixilated image by filtering out pixels of the first pixilated image that do not have an 
intensity exceeding a second threshold value, the second binarized image having 
substantially equal area and coordinate locations as the first binarized image, the 
second binarized image representing a second planar distance from the interactive 
side of the surface; 
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(iv) detecting a first connected component in the first 
binarized image, the first connected component representing a first set of pixels that 
have an intensity exceeding the first threshold value and are immediately adjacent to 
each other, without an intervening region of pixels that do not have an intensity 
exceeding the first threshold value; 

(v) determining a first bounding area and a first coordinate 
location of the first connected component in the first binarized image; 

(vi) detecting a second connected component in the second 
binarized image, the second connected component representing a second set of pixels 
that have an intensity exceeding the second threshold value and are immediately 
adjacent to each other, without an intervening region of pixels that do not have an 
intensity exceeding the second threshold value; 

(vii) determining a second bounding area and a second 
coordinate location of the second connected component in the second binarized 
image that has substantially equal area and coordinate locations as the first binarized 
image; and 

(viii) determining that both the first connected component 
and the second connected component correspond to the three-dimensional object 
adjacent to the interactive side of the surface, if one of the first connected component 
and the second connected components is disposed substantially within the bounding 
area of the other of the first connected component and the second connected 
component. 

12. The system of Claim 11, wherein the machine language instructions 
further cause the processor to: 

(a) associate a first label with the first connected component; 

(b) associate a second label with the second connected component; 

and 

(c) associate the first label with the second label to indicate that 
both the first connected component and the second connected component correspond 
to the three-dimensional object that is adjacent to the interactive side of the surface. 
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13. The system of Claim 11, wherein after placing the three-dimensional 
object on the surface prior to creating the first pixilated image, so that the first planar 
distance is substantially zero, and after creating the first pixilated image, the machine 
language instructions fiirther cause the processor to determine a touch threshold value 
for pixels that should be included in the first pixilated image, corresponding to an 
intensity of the infrared light reflected from a portion of the three-dimensional object 
and received by the light sensor when said portion of the three-dimensional object is 
contacting the surface. 

14. The system of Claim 13, wherein the machine language instructions 
further cause the processor to set the first threshold value to the touch threshold value 
prior to creating the first binarized image, thereby determining whether any portion of 
the three-dimensional object is contacting the surface. 

15. The system of Claim 13, wherein the machine language instructions 
further cause the processor to: 

(a) determine a hover threshold that is different from the touch 
threshold, the hover threshold corresponding to an intensity of the infrared light 
reflected by the three-dimensional object and received by the light sensor when the 
three-dimensional object is a determinable distance from the surface on the 
interactive side of the surface; 

(b) prior to creating the second binarized image, set the second 
threshold value to the hover threshold; and 

(c) determine that the three-dimensional object is at least within a 
hover region away from the surface, the hover region comprising a volume between 
the surface and a plane that is parallel to the surface and is separated from the sxirface 
on the interactive side of the surface by the determinable distance. 
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16, The system of Claim 15, wherein the hover threshold is determined 
prior to creating the first binarized image and the second binarized image, and 
wherein the hover threshold is determined by: 

(a) moving the three-dimensional object toward the surface on the 
interactive side of the surface from a distance sufficiently far from the surface so that 
an intensity of the infrared light reflected from the three-dimensional object and 
received by the Ught sensor initially is not above the first threshold value; 

(b) creating a series of pixilated images over time as the 
three-dimensional object is moved toward the surface; 

(c) detecting a plurality of calibration connected components, 
including one calibration connected component corresponding to the 
three-dimensional object for each of the series of pixilated images, each calibration 
cormected component comprising a set of immediately adjacent pixels with a 
corresponding minimum pixel intensity for the infrared light received by the light 
sensor that is greater than zero for each of the sequence of pixilated images; 

(d) determining which one of the plurality of calibration connected 
components has a maximum bounding area; and 

(e) storing as the hover threshold the corresponding minimum 
pixel intensity that was determined for said one of the plurality of calibration 
connected components that has the maximum bounding area. 

17. The system of Claim 15, wherein the determinable distance is a 
function of an infrared light reflectivity property of the three-dimensional object. 
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18. The system of Claim 15, wherein the machine language instructions 
further cause the processor to: 

(a) create a second pixilated image an interval of time after 
creating the first pixilated image; 

(b) create a third binarized image from the second pixilated image 
by filtering out pixels of the second binarized image that do not have an intensity 
exceeding the hover threshold, so that the second binarized image and the third 
binarized image comprise a sequence of binarized images at the hover threshold; 

(c) detect a third connected component in the third binarized 

image; 

(d) determine at least one of a third bounding area and a third 
location of the third connected component in the third binarized image; and 

(e) compute at least one of a current distance from the surface, a 
change in distance, and a velocity of the three-dimensional object as a function of at 
least two of the second bounding area, the third bounding area, the second location, 
and the third location. 

19. The system of Claim 18, wherein the machine language instructions 
further cause the processor to: 

(a) predict whether the three-dimensional object will contact the 
surface within a predefined period; and 

(b) produce an indication that the three-dimensional object has 
already contacted the surface if the prediction indicates that the three-dimensional 
object will contact the surface within the predefined period, thereby reducing latency 
in providing the indication that the three-dimensional object has contacted the 
surface. 
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20. The system of Claim 15, wherein the machine language instructions 
further cause the processor to respond to a determination that the three-dimensional 
object is at least within the hover region over a specific predefined portion of the 
surface, by displaying a predetermined graphic image at the specific predefined 
portion of the surface, enabling a user to contact a selection within the predetermined 
graphic image with the three-dimensional object, to select an option displayed there 
on the siuface. 
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